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[57] ABSTRACT 

A pattern recognition system for recognizing an un- 
known pattern comprised of symbols which are part of 
a pattern system which is devoid of inherent context 
such as numbers. Artificial contextual information 
based on other than symbol features and the pattern 
system and in the form of probability weighted ex- 
pected interpretations are stored and used in the pro- 
cessing phase of recognition. In the system disclosed, 
the system comprises a neural network whose forward 
and feedback paths are controlled by the output cells of 
the network based, in part, on the contextual informa- 
tion. 
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: symbol is, therefore, categorized by the system accord- 

NEURAL NETWORK MODEL IN PATTERN ing to its essential features. After learning, each gnostic 

RECOGNITION USING PROBABILISTIC cell of the network is, in turn, associated by the user 

CONTEXTUAL INFORMATION with some specific category name, which is a known 

5 common identifier of the symbols whose essential fea- 

B ACKGROUND OF THE INVENTION tures have been associated with the cell during learning. 

This invention relates to pattern recognition systems Thus, for example, where numbers are being recog- 

and, in particular, to pattern recognition systems which nized, the category names would be the known numbers 

employ context and expected values in the recognition 0-9 indicative of symbols having the essential features 

process. 10 of these known numbers. 

In a variety of pattern recognition systems in use The Fukushima network thus learns to categorize 
today, the inherent or natural context of the patterns presented symbols, i.e.; to develop a correspondence 
being recognized is used to aid the recognition system in between individual symbols and associated output cells, 
making its recognition decision. Patterns are said to Accordingly, when an unknown pattern (which con- 
have inherent context if there are natural rules or crite- 15 sists of a sequence of one or more symbols) is presented 
ria governing the patterns and the interrelationships to the network during its recognition period, the pattern 
between patterns or groups of patterns, A particular causes the network to develop a sequence of steady 
example of patterns having such inherent context are states in each of Which there is a maximum output at one 
words and groups of words as in a sentence. In word of the output cells. In each case an unknown symbol is 
recognition systems, the semantic and syntactic rules of 20 therefore recognized by the network as having the es- 
usage of the language provide inherent context for help- sential symbol features corresponding to this output cell 
ing to recognize the individual letters, words and ^ therefore, is identifiable by its category name, 
groups of words. In robotic object recognition systems, During recognition by the Fukushima network, Le., 
the natural order of the world provides inherent con- from presenting a pattern to final steady state (i.e., 
text For example, the laws of physics and knowledge * ^ ^ for ^ ^ ^^^^ g bol of ^ 
about the type of scene (e.g., urban, battlefield, factory, mem% whicheV er output cell has a maximum output 
etc.) ass*t m the reception of objects and groups of at ^ ven ^ ^ cffcrcnt aths of ^ 
objects. In both cases, *e higher- evd inherent or natu- networt ^ ^ a^p^ed by ^ 0 f a maximum 
raJ conte^ial information can ful in the missing lower ^ detector which outputs ; f ^ gn 0S ti c ccUs 0 

While the inherent context of certain patterns has <*ete* toina^^ 

thus proved helpful in recognizing these patterns, there <?*rolled * ^ detector signals via couphng tfiese 

areinlmytypesrfpatterns^ J?* 0 * »W*P«* * ™ ** *> facUitate cells of 

ent context For these patterns there has been no use of he net ?? rk ™& ^ 

context in the recognition processes a result, recog- 35 turcs of the maximum output cell and to mhibit cells of 

nizing these patterns is more difficult and time consum- * e network which are not associated with these symbol 

ing than would otherwise be the case if context were ^ attention of the network thus becomes 

use ^ directed to particular symbol features and this facihtat- 

Digits, alphanumerics or numbers, are types of pat- m 8 md inhibiting effect continues until the network is 

terns without natural or inherent context Thus, in at- 40 t0 reaches its decision. During 

tempting to recognize alphanumerics there is no higher this processing phase, the symbol features upon which 

level meaning or set of rules governing the relationship attention is focused thus become an integral part of the 

of the individual numbers or sets of numbers which iterative recognition process. 

would be useful in the recognition process. Recognition While not present in the Fukishima system, there are 

therefore, is more difficult than would otherwise be the 45 oth er types of systems which upon completion of the 

case if inherent context were present processing phase enter a post-processing phase. In this 

Systems are available, however, for number and phase the decision reached by the recognition system is 

other symbol recognition. A particularly advantageous either accepted or rejected in dependence upon com- 

system is the one disclosed in an Article entitled "A parison of the . recognized pattern with an expected 

Neural Network Model for Selective Attention in Vi- 50 pattern. One system of this type has been proposed for 

sual Pattern Recognition," written by Kunihiko Fuku- reading the digits corresponding to the dollar amounts 

shima and published in Biological Cybernetics, 55, 5-15 written on bank checks received in payment of bills. In 

(1986). This article in its entirety is incorporated herein this system, the previously recognized value is com- 

by reference. pared with a billed amount 

As illustrated in FIG. 1 of the Fukushima article, the 55 While the recognition systems of the prior art and, in 
pattern recognition system of Fukushima is a so-called particular, the Fukushima system, are useable pattern 
"hierarchical neural network" having both forward recognition systems, there still is a need for a pattern 
("afferent") and feedback ("efferent") connections be- recognition system which can more quickly and reliably 
tween and within the stages (Uo, Ui, TJ2, U3) of multiple recognize patterns, especially multiple symbol patterns 
layers of the neural cells (e.g., in the forward path, the 60 and severely deformed patterns, 
cell Uco of the stage Uo and, in the feedback path, the It is, therefore, a primary object of the present inven- 
cell Wgo in the first stage Uo and the cell W c j in the last tion to provide such a pattern recognition system, 
stage U3) of the network. These connections are It is a further object of the present invention to pro- 
weighted or adjusted by the network during a learning vide such a pattern recognition system utilizing derived 
period of the system so that symbols having different 65 contextual information for the patterns to be recog- 
essential features result in a maximum output at differ- nized. 

ent output cells (Le., so called "gnostic cells" Uc3) in the It is yet a further object of the present invention to 

highest level or output stage U3 of the network. Each provide such a pattern recognition system utilizing ex- 
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pected interpretations of the patterns to be recognized is fed via, for example, a pixel matrix, an unknown pat- 

and associated probabilities. tern 3 and to which is also fed contextual information 

It is still a further object of the present invention to from an external data base 4, The recognizer 2 corn- 
provide such a pattern recognition system for recogniz- prises, a pattern modifier 5 whose output feeds a neural 

ing any symbol and, in particular, typed or hand written S network 6 which communicates with a category selec- 

mimbers. tor 7. As shown, the selector 7 is in the form of a rule 

It is also an object of the present invention to provide based category selector, 

such a pattern recognition system for recognizing num- In the present illustrative case, each pattern to be 

hers on checks used in payment of bills. recognized is a sequence N of individual handwritten or 

c TI wvr ABY op thp TMvtJArrTriM 10 P 1 ^^ numbers (see FIG. 2) which might appear in the 

SUMMARY OF THE INVENTION courtesy amount region 21 of a check 22 usedin pay- 

In accordance with the principles of the present in- ment of a bill such as, for example, a telephone bill. As 
vention, the above and other objectives are realized in a illustrated, the data base 4 stores three expected inter- 
pattern recognition system wherein probability pretations (i.e., likely values) Ni, N2, N3 of the pres- 
weighted expected interpretations of an unknown pat- 15 ented pattern (see, FIG. 2), one for each number se- 
tern are utilized as a form of artificial or derived context quence corresponding to a current bill amount, a prior 
(as opposed to inherent or natural context) in the pro- bill amount and the sum of these two bill amounts, with 
cessing (as opposed to post-processing) phase of system corresponding weightings or probabilities Pi, P2, P3. 
operation. By using these weighted expected interpreta- These probabilities may be derived from statistics main- 
tions during processing (i.e., integrally to the recogni- 20 tained by the billing party with respect to checks re- 
tion task), the system is more able to readily, reliably ceived in payment of current bills and along with the 
and quickly recognize unknown patterns. expected interpretations serve as a form of artificial or 

In the form of the invention to be disclosed hereinaf- derived contextual information for the unknown pat- 
ter, the recognition system is a modified type of the tern. As above-indicated, it is this derived contextual 
basic system disclosed in the Fukushima article. In par- 25 information which is integrally used during the process- 
ticular, the Fukushima system is modified so that in- ing phase of the recognizer 2 to achieve recognition, 
stead of controlling the efferent paths during the pro- As depicted in FIG. 1, the neural network 6 is as- 
cessing phase based upon the output cell with maximum sumed to be in FIG. 1 of the Fukushima article. The 
output, control is effected based upon the gnostic cell parameters U C 3, W c3> Wco and X thus correspond to 
outputs as well as the probability weighted expected 30 those parameters as described above. More particularly, 
interpretations. This information is processed in a ruled the network 6 is assumed to include the components of 
based category selector utilizing certainty factors to the Fukushima type, absent the maximum detector 
select a gnostic cell (i.&, to control the efferent or feed- which is used to control the efferent paths of the net- 
back paths). work during the processing phase. As above-indicated, 

In a further aspect of the present invention, where the 35 in the present invention, such control of the efferent 

pattern comprises a sequence of symbols and the proba- paths is effected utilizing the rule-based category selec- 

bility weighted expected interpretations relate to the tor 7 based upon the rules and the probability weighted 

pattern, the category selector is also utilized to update expected interpretations or values (derived contextual 

the certainty factors associated with the probability information) stored in external data base 4. 
weightings of the expected interpretations as symbols of 40 While an extensive explanation of the Fukushima 

the sequence are recognized. These updated values are type neural network has been is set forth above, a brief 

then used by the selector in determining the output cell summary of the operation of the network will again be 

to be used for feedback control. given to facilitate understanding of the network as mod- 

In yet a further aspect of the present invention, once ified in accordance with the present invention. More 

a particular symbol in a pattern is recognized, the cate- 45 particularly, when an unknown pattern to be recog- 

gory selector addresses an input pattern modifier which nized (such as the number N of FIG. 2) is presented to 

acts to delete the recognized symbol from the input the neural network, the attention of the network be- 

making it easier to recognize the remaining symbols. comes initially directed to one of the symbols of the 

Also, means is provided for automatically shifting the pattern. As a result, one of the output or gnostic cells 

attention of the system from an already recognized 50 U C 3 of the network takes on a mnximmn value. This 

symbol in a pattern so that the system moves on to amounts to a preliminary or partial recognition by the 

recognize the remaining symbols. network that the symbol has the essential symbol fea- 

BRIEF DESCRIPTION OF THE DRAWING ^ associated with maximum cell and, hence, can 

be identified with its category name. It is partial recog- 

The above and other features and aspects of the pres- 55 nition because it is made even if the recognition has a 

ent invention will become more apparent upon reading low confidence level, i.c, the maximum output value is 

the following detailed description in conjunction with a relatively low maximum . 

the accompanying drawings, in which: The aforesaid maximum value is then detected by the 

FIG. 1 shows a recognition system in accordance maxim um detector and the fnarimnm detector, in turn, 

with the principles of the present invention; and 60 communicates this information to the efferent or feed- 

FIG. 2 illustrates a number pattern and a correspond- back paths of the network, i.e., to the cells W C 3 in the 

ing set of probability weighted expected interpretations feedback path of the last module U3. The feedback 

for this pattern. paths respond to this excitation by facilitating the re- 

DET AILED DESCRIPTION sponse of those cells in the afferent or forward path of 

65 the network which led to the preliminarily recognized 

In FIG. 1, a recognition system 1 in accordance with symbol and inhibiting the response of other cells. This 

the principles of the present invention is shown. The process iteratively continues as the network increases 

recognition system 1 comprises a recognizer 2 to which its confidence level (as evidenced by an increase in the 
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maximum value) until the network reaches steady state. 
If the network is sufficiently confident at this time (i.e., 
if the maximum value exceeds a predetermined thresh- 
old value), the unknown symbol is recognized as the 
category name of the gnostic cell with maximum out- 5 
put. 

Once this occurs, the network is interrupted and its 
attention then becomes focused on another of the un- 
known symbols of the unknown pattern. The above 
procedure repeats and this unknown symbol is recog- 10 
nized. The remaining unknown symbols are then also 
recognized in similar fashion. 

While the neural network 6 of the present invention 
functions as just described, the determination of which 
of the gnostic cells of the network should control the 15 
feedback paths of the network during processing is now 
under the control of the rule based category selector 7. 
The rule based category selector 7 monitors the outputs 
of all the gnostic cells and receives contextual informa- 
tion from the data base 4. Based on the original proba- 20 
bility weighted expected interpretations or values of the 
unknown pattern (the derived context) and on a set of 
rules of the selector which modify the certainty associ- 
ated with each expected value as the outputs of the 
gnostic cells change and individual symbols are recog- 25 
nized, the rule-based category selector selects which 
gnostic cell should control the efferent or feedback 
paths at each stage of the processing phase and for each 
symbol in the pattern. As a result of this integral use of 
the additional probabilistic information (the derived 30 
context) by the selector in controlling the feedback 
paths, the neural network 6 will be more efficiently, 
more quickly and more certainly driven to steady state 
or convergence for each symbol being recognized. Op- 
eration of the network will thus be greatly improved. 35 
The network, moreover, win now also be able to recog- 
nize severely deformed and/or ambiguous symbols or 
sequences of symbols which it would not have other- 
wise been able to recognize. 

In further accord with the invention, the category 40 
selector 7 is additionally adapted to store information 
concerning the particular symbols in an unknown pat- 
tern as these symbols become recognized. The selector 
7, furthermore, is also able to determine and store the 
entire bit pattern of a recognized symbol (through re- 45 
ceipt of the input Wco), along with its average horizon- 
tal bit position. 

The stored information on the recognized symbols is 
used by the selector 7 to update the certainties or proba- 
bilities of the expected values of the unknown pattern. 50 
Additionally, once a symbol in a pattern has been recog- 
nized, the stored bit pattern of the recognized symbol 
and the average horizontal bit position of the recog- 
nized symbol are used by the selector 7 to develop 
signal m for modifying the unknown input pattern. 55 
Selector 7 is also used to detect steady state and to 
develop signal a for automatically interrupting the net- 
work. 

The signal m is first generated by the selector 7 and it 
instructs the pattern modifier 5 to delete from the un- 60 
known input, the symbol bits just recognized. The sig- 
nal a, in turn, is then generated to interrupt the neural 
network 6 so that the network changes its attention 
from a just recognized symbol to another not yet recog- 
nized symbol in the sequence. 65 

While the selector 7 is illustrated in the present em- 
bodiment as a rule based category selector, it can also be 
an other known type of selector having software 



6 

adapted to function as above-described. Of course, the 
particular decision criteria or rules which the selector 
utilizes to process the gnostic cell outputs and the prob- 
ability weighted expected values will depend upon each 
particular application, including the nature of the pat- 
terns being recognized. One type of rule-based selector 
which might be used is a selector based oh an expert 
system shell using certainty factor calculus. A typical 
expert system shell utilizing one such certainty factor 
calculus is one manufactured by Teknowledge under 
the brand name M M.l.". 

A rule based system used for the selector 7 might 
typically comprise three parts: an inference engine, a 
rule set (a collection of rules of the form "IF A and B 
and C then D") and a cache (or database of known facts, 
each of which is known with some certainty). The sys- 
tem would operate in the following way. Given an 
initial set of facts,' associated certainties and a goal (e.g., 
"select a gnostic cell"), the inference engine searches 
the rule set for rules which will help it to achieve the 
goal by deriving a new fact from already known facts or 
from facts which can be derived from known facts and 
the rule set. If all the premises in a rule are known with 
some certainty, the rule is satisfied and its concluding 
clause asserted as a new fact The certainty factors of 
the clauses of the rule are disjunctively and/or conjunc- 
tively combined using an established certainty factor 
calculus to obtain a combined certainty factor which is 
multiplied by the certainty factor associated with the 
rule itself to derive a certainty factor for the newly 
derived fact. If the same fact is asserted more than once, 
the certainties of the separate assertions are combined to 
reflect the corroborating information. Since the goal 
might be achieved in a variety of distinct ways by differ- 
ent combinations of rules and facts, the system selects 
the goal achieved with the highest overall certainty 
factor. 

In the present application, the facts initially in the 
cache would be based upon the outputs of the gnostic 
cells and the expected interpretations of the patterns. 
The certainty factors, in turn, for the gnostic cells 
would depend on the relative magnitudes of the cell 
outputs and those for the expected patterns on the re- 
lated probabilities. 

While, as above-described, the rule set for a particu- 
lar category selector 7 would depend upon each indi- 
vidual situation, an illustrative, simplified set of selec- 
tion rules is given below. Also, application of these rules 
to selection of particular gnostic cells in recognizing the 
number N of FIG. 2 is likewise provided. In applying 
the rules the following govern. 

(1) "X", "Y" and 'XT are variables and the rules 
might be instantiated by several sets of facts in which 
the variables hold the proper relations to one another. 

(2) Each premise has a certainty factor (CF) associ- 
ated with it. This CF is given in the following way for 
various premises: 

(a) The CF for the premise: "the gnostic cell for 
known symbol *X* is on", is the normalized magni- 
tude of the gnostic cell associated with that sym- 
bol. 

(b) The CF for the premise: * M X* is a symbol in ex- 
pected pattern *Y"\ is initially the probability of 'Y* 
as given in the external database 4. This CF will be 
updated by other rules. 

(c) The CF for a Boolean premise is 1 if the premise 
is true and 0 if the premise is false. 
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(d) The CF for any other premise is as determined by goal of selecting a gnostic cell) arc then compared and 

the rule set the goal fact with highest overall certainty is selected. 

The CFs for each rule are assigned using the judg- This goal fact then governs selection of the gnostic cell 

ment and experience of the recognition system designer of the neural network 6 which is to be used to control 

based upon the particular application. j the feedback paths of the network. 

The representative rule set (in English syntax) is as As an example of how the rule set operates, it will be 

Rtt *f 1: applied to identification of the number N of FIG. 2. For 

nntJlfTPT , t ^ t t this example, it will be assumed that the certainty factor 

^r£SF. ; S 1108 * 10 0611 for Ac laQwa 3ym " for RuIc 4, R* fa eq«al to 0.9, and the certainty factors 

ool jC is on; and to f or aU the other rules are equal to 1.0. 

• PI !f 2: " X " " a symbo1 m ex P ected P at - When the number N is first presented to the neural 

-m. e iJL - r w 1 » v „ network 6, let it be assumed that the gnostic cell with 

Then: Select the gnostic ceU for symbol "X". [Cer- the category name turns on with a normalized mag- 

tainty factor for rule is Ri]. nitudc of 0.8. The category selector 7 then evaluates the 

jf? 15 rule set based on this output and the probabilities Pi, P2, 

PttPfcTTCT? 1 . rw ow «k^i ~f u„„ . p 3 (0.5, 0.3, 0.2) of the expected interpretations or values 

^Sei aS P Nl ' N > 311(1 N 3 s*own in FIG. 2. This evaluation will 

FREN^HE* the interpretation of that symbol is 

"X"- and 20 and * tne S 1108 * 10 ceU with m c category name ^ ' being 

PREMISE 3: "X» is a symbol in the pattern «Y"; "J"* ^ ^ («°« for each of the expected 

mi numbers Ni, N2 and N3) with respective individual 

PREMISE 4: "X" is not a symbol in any other **» of 0.5, 0.3 and 0.2. Tliese individual 

expected pattern, certamty factors are determined using the rule certainty 

Then: Increase the probability that the pattern is "Y"; 25 ^ tors ' the premise certainty factors, 

and decrease the probability that the pattern is any m accordance with the following equation, 

other expected patten, [Certainty factor of rule 2 cW-aW-lN {cf,^ (1) for „,„ 

KUie 3: conjunctive 

If: 30 premises. For 

PREMISE 1 : One symbol of the pattern has been "J" with 

recognized; and prS^ 

PREMISE 2: the recognized symbol appears in all rule is split 

expected patterns; into its 

Then: Do not adjust the probabilities of any of the 35 eqwvalent 

expected patterns. [Certainty factor of rule 3 is R 3 ]. cJnj^tiw 

Rule 4: premises. 

If a fact is asserted once, the overall certainty factor 

PREMISE 1: The second symbol of a pattern is CFoverall for that fact is CFjtvd/k. Ifa fact is asserted 

being recognized; and 40 more than once (as above), the corroborating informa- 

PREMISE 2: the gnostic cell for pattern "X" is on; tion reinforces the conclusion. The first time the fact is 

md u asserted, CFoverall is set to CFINDIV. If the same 

PREMISE 3: "X" is a symbol in the expected pat- fact is asserted again based on independent information, 

tern "Y~; and the individual certainty factor for the second assertion 

PREMISE 4: the symbol recognized so far is "C"; 45 CF/w/Kis combined with CFoverall to derive a new 

CFoverall for the asserted fact: 

PREMISE 5; "C" is a symbol in the expected pat- 
tern "Y"; and CF 0 verall(NE W) - CFoverallH i - - 

PREMISE 6: "C" and "X" appear in the sequence CFoverall)* CFwdiv (2) 
in the order that they do in the expected pattern" 50 

(according to their average horizontal bits); Applying the above to the multiple selection of the 

Then: select the gnostic cell for the symbol "X". category name **2" results in the overall certainties as 

[Certainty factor of rule 4 is R4.I follows: 
Rule 5: 

If ; 53 QY OVERALL-^ (0.5)=0.5 [(irst assertion] 

PREMISE 1: none of the rules are satisfied: 

Then: select the gnostic ceU™tn S *agni- CF otm<u = 1 .o ( a5 + (.-o.5X0.3)]-o.« i5 ^ 

tude. [Certainty factor of rule 5 is R5.] 

As above-indicated, certainty factors for the forego- CP overall^ 0 [0.65+0-0.65X0.2)] =0.72 [third assertion] 
ing rule set are applied by first combining the certainty 60 

factors for the premises of a rule that fires and then If these are the only rules relating to the goal of se- 

combining the resultant with the certainty factor for the leering a gnostic cell that are satisfied, the gnostic cell 

rule to obtain an overall certainty factor. This overall and, therefore, symbol with the highest certainty factor 

certainty factor may be altered using the certainty fac- (in this case **2" with certainty factor 0.72) will be se- 

tor calculus by separate assertions of the same fact (cor- 65 lected. The category name or number "2" will thus 

roborating information). When all applicable rules have control the feedback paths of the neural network, 

fired, the overall certainty factors for the resultant de- Under this control when the neural network 6 

rived goal facts (those derived facts which satisfy the reaches steady state, the number "2" will be recognized 
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as being a symbol in the unknown N. At this time, Rule 
2 will not be satisfied because the number <( 2" appears in 
al) expected patterns Ni, N2, N3. As a result, the proba- 
bilities for the expected patterns will not be adjusted. 

Now assume that after recognition of the number 5 
"2", erasure of the associated bit pattern by m and fol- 
lowing the interruption of the neural network 6 by the 
signal a, the network redirects its attention to the num- 
ber "9" causing the corresponding gnostic cell to turn 
on with normalized magnitude 0.7. Assuming further 10 
that the average horizontal bit position of the number 
"9" upon which the attention of the neural network 
module 6 is focused is such that the number "9 " follows 
the number "2" already recognized, Rules 1 and 4 will 
be satisfied with a resultant overall certainty 0.49 calcu- 15 
lated in accordance with the above. 

Other rules (not indicated) may also fire and the gnos- 
tic cell for the goal fact asserted with maximum cer- 
tainty factor will be selected. This will continue until 
the number N is recognized. 20 

As can be appreciated, with recognition system 1 
configured as above, the system exhibits marked advan- 
tages over the Fukushima network since additional 
information is brought to bear on the recognition task. 
Thus, the system has increased speed and accuracy in 25 
carrying out the recognition process, because of the use 
of the derived contextual information in the processing 
phase of the system operation. Also, the ability to rec- 
ognize sequences of symbols or deformed patterns is 
enhanced. 30 

It should be noted that the functions of the system 1 
as illustrated in FIG. 1, can be carried out by a pro- 
grammed computer or microprocessor. Furthermore, 
while the system has been illustrated in terms of recog- 
nizing patterns comprised of number symbols which 35 
correspond to check amounts with associated probabili- 
ties based on current and past bills, the system can be 
used to recognize patterns comprised of all types of 
symbols. Accordingly, as used herein, the term symbol 
is meant in its broadest sense and is intended to include 40 
any and all types of characters, numbers, drawings and 
indicia of any nature. 

In all cases, it is understood that the above-identified 
arrangements are merely illustrative of the many passi- 
ble specific embodiments which represent applications 45 
of the present invention. Numerous and varied other 
arrangements can readily be devised in accordance with 
the principles of the present invention without depart- 
ing from the spirit and scope of the invention* 
What is claimed is: 50 
1. A system for recognizing an unknown pattern, said 
unknown pattern including one or more symbols which 
are part of a pattern system which is devoid of inherent 
context, comprising: 
storing means for storing artificial contextual infor- 55 
mation associated with said unknown pattern and 
based upon other than information as to symbol 
features and the pattern system; and 
recognizer means responsive to said known pattern 
and to said artificial contextual information, said 60 
recognizer means having a processing phase 
wherein said recognizer means processes at least 
one symbol of said unknown pattern and outputs as 
a recognition decision that said one symbol is one 
of a plurality of category names and wherein in 65 
reaching said recognition decision during said pro- 
cessing phase utilizes said artificial contextual in- 
formation. 
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2. A system in accordance with claim 1 wherein: 
said unknown pattern includes a plurality of symbols; 
and said recognizer means during said processing 

phase processes each of said symbols of said un- 
known pattern and outputs as a recognition deci- 
sion for each of said symbols that the symbol is one 
of a plurality of category names and in reaching 
said recognition decision during said processing 
phase utilizes said derived contextual information. 

3. A system in accordance with claim 2 wherein: 
said derived contextual information comprises ex- 
pected interpretations of said unknown pattern and 
probability values associated with such expected 
interpretations. 

4. A system in accordance with claim 3 wherein: 
said unknown pattern is a sequence of individual 

numbers. 

5. A system in 'accordance with claim 4 wherein: 
said numbers correspond to the dollar amount of a 

check. 

6. A system in accordance with claim 5 wherein: 
said check is utilized in payment of a bill. 

7. A system in accordance with claim 6 wherein: 
one of said expected interpretations of said unknown 

pattern corresponds to the dollar amount of said 
bill. 

8. A system in accordance with claim 7 wherein: 
others of said expected interpretations of said un- 
known pattern correspond to the dollar amount of 
a bill previous to said bill and the sum of the dollar 
amounts of said bill and previous bill. 

9. A system in accordance with claim 8 wherein: 
said bills are telephone bills. 

10. A system in accordance with claim 2 wherein: 
said recognizer means comprises: neural network 

means responsive to said unknown pattern for car- 
rying out said processing phase; and selector means 
responsive to said derived contextual information 
for controlling said neural network. 
1L A system in accordance with claim 10 wherein: 
said neural network means comprises: a plurality of ' 
stages of cells including an input stage for receiving 
said unknown pattern and an output stage having 
output cells each corresponding to a different one 
of said category names and to a different set of 
essential symbol features; an afferent path connect- 
ing said stages; and an efferent path connecting said 
stages for facilitating and inhibiting said afferent 
path; and 

said selector means is responsive to said output cells 
and based thereon and on said derived contextual 
information controls said efferent path. 

12. A system in accordance with claim 11 wherein: 
said selector means selects one of said output cells to 

condition said efferent path so said efferent path 
facilitates transmission in said afferent path associ- 
ated with the set of essential symbol features corre- 
sponding to said selected output cell. 

13. A system in accordance with claim 12 wherein: 
said selector means comprises a rule based selector 

operating on the basis of certainty factor calculus. 

14. A system in accordance with claim 10 wherein: 
said recognizer means comprises means for automati- 
cally changing the attention of said neural network 
when said recognizer means outputs a recognition 
decision with respect to a symbol in said pattern. 

15. A system in accordance with claim 2 wherein; 
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said recognizer means comprises means for deleting 
from said unknown pattern the symbols of said 
unknown pattern for which said recognizer means 
has already made a recognition decision. 

16. A system in accordance with claim 2 wherein: 
said recognizer means includes means for adjusting 

said derived contextual information in dependence 
on said recognizer means reaching a recognition 
decision with respect to the individual symbols in 
said pattern. 

17. A method of recognizing an unknown pattern, 
said unknown pattern including one or more symbols 
which are part of a pattern system which is devoid of 
inherent context, comprising: 

storing artificial contextual information associated 
with said unknown pattern and based upon other 
than information as to symbol features and the 
pattern system; and 

recognizing said unknown pattern by processing at 20 
least one symbol of said unknown pattern and out- 
putting as a recognition decision that said one sym- 
bol is one of a plurality of category names utilizing 
said artificial contextual information. 

18. A method in accordance with claim 17 wherein: 
said unknown pattern includes a plurality of symbols; 
and said recognizing includes processing each of said 

symbols of said unknown pattern and outputting a 
recognition decision for each of said symbols that 
the symbol is one of a plurality of category names 
utilizing said derived contextual information. 

19. A method in accordance with claim 18 wherein: 
- said derived contextual information comprises ex- 
pected interpretations of said unknown pattern and 
probability values associated with such expected 
interpretations. 

20. A method in accordance with claim 19 wherein: 
said unknown pattern is a sequence of individual 

numbers. 

21. A method in accordance with claim 20 wherein: 
said numbers correspond to the dollar amount of a 

check. 

22. A method in accordance with claim 21 wherein: 
said check is utilized in payment of a bill. 

23. A method in accordance with claim 22 wherein: 
one of said expected interpretations of said unknown 

pattern corresponds to the dollar amount of said 
bill. 

24. A method in accordance with claim 23 wherein: 30 
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others of said expected interpretations of said un- 
known pattern correspond to the dollar amount of 
a bill previous to said bill and the sum of the dollar 
amounts of said bill and previous bill. 

25. A method in accordance with claim 24 wherein: 
said bills are telephone bills. 

26. A method in accordance with claim 18 wherein: 
said processing step is carried out by neural network 

means responsive to said unknown pattern and by 
selector means responsive to said derived contex- 
tual information for controlling said neural net- 
work. 

27. A method in accordance with claim 26 wherein: 
said neural network means comprises: a plurality of 

stages of cells including an input stage for receiving 
said unknown pattern and an output stage having 
output cells each corresponding to a different one 
of said category names and to a different set of 
essential symbol features; an afferent path connect- 
ing said stages; and an efferent path connecting said 
stages for facilitating and inhibiting said afferent 
path; and 

said selector means is responsive to said output cells 
and based thereon and on said derived contextual 
information and controls said efferent path. 

28. A method in accordance with claim 27 wherein: 
said selector means selects one of said output cells to 

condition said efferent path so said efferent path 
facilitates transmission in said afferent path associ- 
ated with the set of essential symbol features corre- 
sponding to said selected output cell. 

29. A method in accordance with claim 28 wherein: 
said selector means comprises a rule based selector 

operating on the basis of certainty factor calculus. 

30. A method in accordance with claim 26 wherein: 
said recognizing comprises automatically changing 

the attention of said neural network when a recog- 
nition decision with respect to a symbol in said 
pattern is reached. 

31. A method in accordance with claim 18 wherein: 
said recognizing step includes deleting from said un- 
known pattern the symbols of said unknown pat- 
tern for which a recognition decision has already 
been reached. 

32. A method in accordance with claim 18 wherein: 
said recognizing step includes adjusting said derived 

contextual information in dependence on reaching 
a recognition decision with respect to the individ- 
ual symbols in said pattern. 

***** 
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UNITED STATES PATENT AND TRADEMARK OFFICE 

CERTIFICATE OF CORRECTION 

PATENT NO. : 4,876,731 

DATED : October 24, 1989 

INVENTOR(S) : Keith Loris, et al. 

It is certified that error appears in the above-identified patent and that said Letters Patent is hereby 
corrected as shown below: 

Col. 2, line 29, change "o" to — to— . 

Col 2 line 43, change "Fukishima" to -- Fukushima— -. 

Col! A, line 28, after "be" insert — of the Fukushima 

type discussed at length above and shown 

Col. 4, line 32, after "type" insert — network 

Col. 5, line 68, change "an" to — any --. 

Col. 7, line 6, after "as" insert — follows: --. 

Col. 9, line 55, change "storing" to — storage 

Col. 9, line 59, change "known" to — unknown 

Col. 10, line 11, change "derived" to — artificial — . 

Col. 10, line 39, change "derived" to — artificial 

Col. 10, line 52, change "derived" to — artificial 

Col. 10, line 68, change ";" to — : — . _ 

Col. 11, line 7, change "derived" to -- artificial — . 

Col. 11, line 31, change "derived" to — artificial — . 

Col. 11, line 33, change "derived' to — artificial 

Col. 12, line 10, change "derived" to ~ artificial --. 

Col. 12, line 24, change "derived" to — artificial — . 

Col. 12, line 46, change "derived to — artificial 

Signed and Sealed this 
Twenty-seventh Day of November, 1990 



Attest: 

HARRY F. MANBECK, JR. 
Attesting Officer Commissioner of Founts and Trademarks 
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